Evaluating Explanation Methods for Deep Learning in Security

Deep learning is increasingly used as a building block of security systems. Unfortunately, neural networks are hard to interpret and typically opaque to the practitioner. The machine learning community has started to address this problem by developing methods for explaining the predictions of neural networks. While several of these approaches have been successfully applied in the area of computer vision, their application in security has received little attention so far. It is an open question which explanation methods are appropriate for computer security and what requirements they need to satisfy. In this paper, we introduce criteria for comparing and evaluating explanation methods in the context of computer security. These cover general properties, such as the accuracy of explanations, as well as security-focused aspects, such as the completeness, efficiency, and robustness. Based on our criteria, we investigate six popular explanation methods and assess their utility in security systems for malware detection and vulnerability discovery. We observe significant differences between the methods and build on these to derive general recommendations for selecting and applying explanation methods in computer security.Comment: IEEE European Symposium on Security and Privacy, 202

Machine Unlearning of Features and Labels

Removing information from a machine learning model is a non-trivial task that requires to partially revert the training process. This task is unavoidable when sensitive data, such as credit card numbers or passwords, accidentally enter the model and need to be removed afterwards. Recently, different concepts for machine unlearning have been proposed to address this problem. While these approaches are effective in removing individual data points, they do not scale to scenarios where larger groups of features and labels need to be reverted. In this paper, we propose the first method for unlearning features and labels. Our approach builds on the concept of influence functions and realizes unlearning through closed-form updates of model parameters. It enables to adapt the influence of training data on a learning model retrospectively, thereby correcting data leaks and privacy issues. For learning models with strongly convex loss functions, our method provides certified unlearning with theoretical guarantees. For models with non-convex losses, we empirically show that unlearning features and labels is effective and significantly faster than other strategies.Comment: Network and Distributed System Security Symposium (NDSS) 202

Evil from Within: Machine Learning Backdoors through Hardware Trojans

Backdoors pose a serious threat to machine learning, as they can compromise the integrity of security-critical systems, such as self-driving cars. While different defenses have been proposed to address this threat, they all rely on the assumption that the hardware on which the learning models are executed during inference is trusted. In this paper, we challenge this assumption and introduce a backdoor attack that completely resides within a common hardware accelerator for machine learning. Outside of the accelerator, neither the learning model nor the software is manipulated, so that current defenses fail. To make this attack practical, we overcome two challenges: First, as memory on a hardware accelerator is severely limited, we introduce the concept of a minimal backdoor that deviates as little as possible from the original model and is activated by replacing a few model parameters only. Second, we develop a configurable hardware trojan that can be provisioned with the backdoor and performs a replacement only when the specific target model is processed. We demonstrate the practical feasibility of our attack by implanting our hardware trojan into the Xilinx Vitis AI DPU, a commercial machine-learning accelerator. We configure the trojan with a minimal backdoor for a traffic-sign recognition system. The backdoor replaces only 30 (0.069%) model parameters, yet it reliably manipulates the recognition once the input contains a backdoor trigger. Our attack expands the hardware circuit of the accelerator by 0.24% and induces no run-time overhead, rendering a detection hardly possible. Given the complex and highly distributed manufacturing process of current hardware, our work points to a new threat in machine learning that is inaccessible to current security mechanisms and calls for hardware to be manufactured only in fully trusted environments

Probing interneuronal cell communication via optogenetic stimulation

This study uses an all-optical approach to probe interneuronal communication between spiral ganglion neurons (SGNs) and neurons of other functional units, in this case cortex neurons (CNs) and hippocampus neurons (HNs), for the first time. We combined a channelrhodopsin variant (CheRiff) with a red genetically encoded calcium indicator (jRCaMP1a), enabling simultaneous optical stimulation and recording from spatially separated small neuronal populations. Stimulation of SGNs was possible with both optogenetic manipulated HNs and CNs, respectively. Furthermore, a dependency on the pulse duration of the stimulating light in regard to the evoked calcium response in the SGNs was also observed. Our results pave the way to enable innovative technologies based on “biohybrid” systems utilizing the functional interaction between different biological (eg, neural) systems. This can enable improved treatment of neurological and sensorineural disorders such as hearing loss

Real-Time Radar-Based Gesture Detection and Recognition Built in an Edge-Computing Platform

In this paper, a real-time signal processing frame-work based on a 60 GHz frequency-modulated continuous wave (FMCW) radar system to recognize gestures is proposed. In order to improve the robustness of the radar-based gesture recognition system, the proposed framework extracts a comprehensive hand profile, including range, Doppler, azimuth and elevation, over multiple measurement-cycles and encodes them into a feature cube. Rather than feeding the range-Doppler spectrum sequence into a deep convolutional neural network (CNN) connected with recurrent neural networks, the proposed framework takes the aforementioned feature cube as input of a shallow CNN for gesture recognition to reduce the computational complexity. In addition, we develop a hand activity detection (HAD) algorithm to automatize the detection of gestures in real-time case. The proposed HAD can capture the time-stamp at which a gesture finishes and feeds the hand profile of all the relevant measurement-cycles before this time-stamp into the CNN with low latency. Since the proposed framework is able to detect and classify gestures at limited computational cost, it could be deployed in an edge-computing platform for real-time applications, whose performance is notedly inferior to a state-of-the-art personal computer. The experimental results show that the proposed framework has the capability of classifying 12 gestures in real-time with a high F1-score.Comment: Accepted for publication in IEEE Sensors Journal. A video is available on https://youtu.be/IR5NnZvZBL

Dos and Don'ts of Machine Learning in Computer Security

With the growing processing power of computing systems and the increasing availability of massive datasets, machine learning algorithms have led to major breakthroughs in many different areas. This development has influenced computer security, spawning a series of work on learning-based security systems, such as for malware detection, vulnerability discovery, and binary code analysis. Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance and render learning-based systems potentially unsuitable for security tasks and practical deployment. In this paper, we look at this problem with critical eyes. First, we identify common pitfalls in the design, implementation, and evaluation of learning-based security systems. We conduct a study of 30 papers from top-tier security conferences within the past 10 years, confirming that these pitfalls are widespread in the current security literature. In an empirical analysis, we further demonstrate how individual pitfalls can lead to unrealistic performance and interpretations, obstructing the understanding of the security problem at hand. As a remedy, we propose actionable recommendations to support researchers in avoiding or mitigating the pitfalls where possible. Furthermore, we identify open problems when applying machine learning in security and provide directions for further research.Comment: to appear at USENIX Security Symposium 202

Lessons Learned on Machine Learning for Computer Security

We identify 10 generic pitfalls that can affect the experimental outcome of AI driven solutions in computer security. We find that they are prevalent in the literature and provide recommendations for overcoming them in the future

Scanning laser optical tomography for in toto imaging of the murine cochlea

The mammalian cochlea is a complex macroscopic structure due to its helical shape and the microscopic arrangements of the individual layers of cells. To improve the outcomes of hearing restoration in deaf patients, it is important to understand the anatomic structure and composition of the cochlea ex vivo. Hitherto, only one histological technique based on confocal laser scanning microscopy and optical clearing has been developed for in toto optical imaging of the murine cochlea. However, with a growing size of the specimen, e.g., human cochlea, this technique reaches its limitations. Here, we demonstrate scanning laser optical tomography (SLOT) as a valuable imaging technique to visualize the murine cochlea in toto without any physical slicing. This technique can also be applied in larger specimens up to cm3 such as the human cochlea. Furthermore, immunolabeling allows visualization of inner hair cells (otoferlin) or spiral ganglion cells (neurofilament) within the whole cochlea. After image reconstruction, the 3D dataset was used for digital segmentation of the labeled region. As a result, quantitative analysis of position, length and curvature of the labeled region was possible. This is of high interest in order to understand the interaction of cochlear implants (CI) and cells in more detail. © 2017 Nolte et al.This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.DFG/EXC/1077/1Ministry of Lower SaxonyVolkswagenStiftun

Oropharyngeal dysphagia in older persons - from pathophysiology to adequate intervention : a review and summary of an international expert meeting

Oropharyngeal dysphagia (OD) is a highly prevalent and growing condition in the older population. Although OD may cause very severe complications, it is often not detected, explored, and treated. Older patients are frequently unaware of their swallowing dysfunction which is one of the reasons why the consequences of OD, ie, aspiration, dehydration, and malnutrition, are regularly not attributed to dysphagia. Older patients are particularly vulnerable to dysphagia because multiple age-related changes increase the risk of dysphagia. Physicians in charge of older patients should be aware that malnutrition, dehydration, and pneumonia are frequently caused by (unrecognized) dysphagia. The diagnosis is particularly difficult in the case of silent aspiration. In addition to numerous screening tools, videofluoroscopy was the traditional gold standard of diagnosing OD. Recently, the fiberoptic endoscopic evaluation of swallowing is increasingly utilized because it has several advantages. Besides making a diagnosis, fiberoptic endoscopic evaluation of swallowing is applied to evaluate the effectiveness of therapeutic maneuvers and texture modification of food and liquids. In addition to swallowing training and nutritional interventions, newer rehabilitation approaches of stimulation techniques are showing promise and may significantly impact future treatment strategies

Population Health Science: A Core Element of Health Science Education in Sub-Saharan Africa

Sub-Saharan Africa suffers an inordinate burden of disease and does not have the numbers of suitably trained health care workers to address this challenge. New concepts in health sciences education are needed to offer alternatives to current training approaches. A perspective of integrated training in population health for undergraduate medical and nursing education is advanced, rather than continuing to take separate approaches for clinical and public health education. Population health science educates students in the social and environmental origins of disease, thus complementing disease-specific training and providing opportunities for learners to take the perspective of the community as a critical part of their education. Many of the recent initiatives in health science education in sub-Saharan Africa are reviewed, and two case studies of innovative change in undergraduate medical education are presented that begin to incorporate such population health thinking. The focus is on East Africa, one of the most rapidly growing economies in sub-Saharan Africa where opportunities for change in health science education are opening. The authors conclude that a focus on population health is a timely and effective way for enhancing training of health care professionals to reduce the burden of disease in sub-Saharan Africa